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@ Method per performing special effects in the receiver of a coded moving picture transmission 
system. 



(57) A moving picture is transmitted without bring- 
ing about a decline in compression efficiency 
and a deterioration in picture quality due to 
coding/decoding. On the basis of effect-desig- 
nating data (313) that has been received, a 
terminal on the receiving side perfonms the 
designated effect on the picture data which has 
been obtained by decoding coded data received 
from a common transmission line (S19), and 
displays the edited image data on a display unit. 
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This invention relates to a nnethod and apparatus 
formoving-picture processing and. more particularly, 
to a moving-picture processing method and appara- 
tus for subjecting a moving picture to such editing as 
superimposing processing, a transition effect, etc. 5 

When digital moving-picture data is edited, com- 
pressed and transmitted in the prior art, the general 
practice is to edit the moving-picture data and then 
compress the moving-picture data that has been 
processed. It should be noted that the term "editing" io 
as used here means superimposing a character, fig- 
ure or the like on a moving picture or subjecting a 
moving picture to a transition effect, described below. 

The transition effect is processing for the purpose 
of smoothly connecting two moving-picture sequenc- is 
es that are continuous in terms of time and involves 
causing the moving picture that is earlier in time to 
fade out while causing the moving picture that is later 
in time to fade in. In this case, an image having a spe- 
cial state in which the two moving-picture sequences 20 
are superimposed appears within a range designated 
in a transition interval. 

The edited moving-picture data thus obtained is 
coded by any moving-picture coding method avail- 
able and the coding data is transmitted to the receiv- 25 
ing side through a communication line. The coded 
data received is decoded and displayed on the receiv- 
ing side. 

Figs. 2A, 2B are flowcharts illustrating proce- 
dures for editing moving pictures according to the pri- 30 
or art, in which Fig. 2A shows the procedure per- 
formed by the terminal on the transmitting side and 
Fig. 2B the procedure performed by the terminal on 
t be texeiyj nQ3id e.. 



According to the flowchart shown in Fig, 2A, the 35 
terminal on the transmitting side enters the moving- 
picture data and stores it in a memory at step S1. 
Next, the terminal on the transmitting side accepts an 
effect designation, such as a designation of the tran- 
sition effect, at step S2, after which the terminal edits 40 
the moving-picture data, which has been stored in the 
memory, on the basis of the designation accepted at 
step S2. This is followed by step S4, at which the ter- 
minal on the transmitting side codes the edited mov- 
ing-picture data, and then by step S5, at which the 45 
terminal transmits the coded data. 

Next, according to the flowchart shown in Fig. 
2B, the terminal on the receiving side receives the 
coded data at step S6. Then, at step S7, the terminal 
on the receiving side decodes the coded data re- so 
ceived and stores the moving-picture data thus de- 
coded in a memory. The terminal on the receiving 
side displays the moving picture on a display unit at 
step S8. 

However, the technique described above in- 55 
volves a number of problems. 

Specifically, in the above-described technique, 
the edited image data that has been subjected to the 



transition effect or the like is coded. However, in the 
conventional moving-picture coding method that per- 
forms compression utilizing the correlation of mutual- 
ly adjacent frames, the frame correlation is lost and 
efficient compression is difficult when it is attempted 
to compress special image data that has been sub- 
jected to the transition effect or the like. 

Further, according to some moving-picture cod- 
ing methods, data is transmitted upon being thinned 
out and coded at a frame rate of ten frames per sec- 
ond in order to achieve a rate of compression. How- 
ever, in the case of a moving picture having a low 
frame rate, a smooth transition effect is difficult to re- 
alize and the transition of the picture becomes too jer- 
ky. 

In display of superimposed captions in moving 
pictures such as television pictures and movies, often 
the color white, of some brightness, is used for the 
captions in order to enhance readability. However, in 
the case of a compression method based upon quan- 
tization in a frequency region according to the MPEG 
standard or the like, mosquito noise is produced at 
the periphery of the characters when a moving pic- 
ture having a caption superimposed on it is com- 
pressed. In addition, owing to concentration of bits in 
the character portions, the quality of the overall pic- 
ture deteriorates (in comparison with a picture not 
having superimposed captions). 

Furthermore, there are many problems in terms 
of operability as far as the user is concerned. For ex- 
ample, the position of a caption usually is fixed at the 
bottom of the screen and the display position cannot 
be changed at will. In addition, when a caption be- 
wLOAOij,as^u. i|j lae^ G^e^s^s ^a r^y /yaiiGh a ng.eQy.eritGa n m otibewmad e>^ 
to a mode in which the caption is not displayed. 

Another problem is that a changeover cannot be 
made between Japanese and English captions when 
viewing captions in a moving picture such as a movie. 
Furthermore, when two kinds of captions are re- 
quired, it is necessary for the user to acquire two 
types of moving-picture data that differ only in terms 
of their captions. Moreover, the producer is required 
to produce different editions depending upon the 
sales area (English-speaking areas, Japanese- 
speaking areas, etc). This involves considerable 
waste. 

An object of the present invention is to address 
the foregoing drawbacks of the prior art. 

More specif ically, an object of the present inven- 
tion is to prevent a decline in compression rate in code 
and uncoding as well as a deterioration in picture 
quality. 

According to a first aspect of the invention, there 
is provided a moving picture processing apparatus 
comprising receiving means for receiving coded digi- 
tal moving-picture data and command data for com- 
manding effect processing to be applied to the digital 
moving-picture data, decoding means for decoding 
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the coded digital moving- picture data, processing 
means for subjecting the digital moving-picture data 
decoded by the decoding means to effect processing, 
and image forming means for forming an image in de- 
pendence upon the digital moving- picture data proc- 5 
essed by the processing means. The digital moving- 
picture data and the command data are transmitted 
by a common transmission line. 

Another object of the present invention is to real- 
ize the transition effect efficiently and with excellent io 
image quality. 

A further object of the present invention is to fully 
perform a superimposing function. 

Other features and advantages of the present in- 
vention will be apparent from t he following description is 
taken in conjunction with the accompanying draw- 
ings, in which like reference characters designate the 
same or similar parts throughput the figures thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 20 

Fig. 1 is a block diagram illustrating an example 
of the configuration of a system for processing a 
moving picture by a moving-picture processing 
method according to a first embodiment of the 25 
present invention; 

Figs. 2A and 2B are flowcharts illustrating proce- 
dures for editing a moving picture according to 
the prior art; 

Figs. 3A and 38 are flowcharts illustrating proce- 30 

dures for editing a moving picture according to 

the first embodiment of the present invention; 

Fig. 4 is a block diagram of a second embodiment 

according to the present invention; 

Fig. 5 is a block diagram of a third embodiment 35 

according to the present invention; 

Fig. 6A is a diagram illustrating a fade-out proc- 

essing according to a first embodiment of the 

present invention; and 

Fig. 68 is a diagram illustrating a conventional 40 
fade-out processing. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

45 

Methods of processing a moving picture accord- 
ing to embodiments of the present invention will now 
be described in detail with reference to the drawings. 

<First Embodiment> so 

Fig. 1 is a block diagram illustrating an example 
of the configuration of a system for processing a mov- 
ing picture according to a first embodiment of the 
present invention. 55 

The system shown in Fig. 1 is composed of a plur- 
ality of terminals a1, b2, c3, ... interconnected by a 
communication line 101. The terminal a1 is construct- 



ed as set forth below. 

Specifically, the terminal al includes an input port 
11 for accepting moving-picture data from an image 
input unit 19 such as a video camera, and a CPU 12 
for executing the processing of image data in accor- 
dance with a control program stored in a ROM 14 as 
well as overall control of the terminal al via a bus 18. 
Further, by accessing a prescribed area in a RAM 13, 
the CPU 12 is capable of reading out a pixel value at 
any coordinates of any frame of the moving-picture 
data stored in the RAM 13. 

The terminal al further includes a coding/decod- 
ing circuit 1 5. a display 1 6 and an interface 1 7 through 
which data communication is performed with the 
communication line 101. 

The aforementioned components of the terminal 
al are interconnected by a bus 18. 

The terminals b2, c3, ... have substantially the 
same construction as that of the terminal al. 

Figs. 3A and 38 are flowcharts illustrating proce- 
dures for editing a moving' picture according to the 
first embodiment of the present invention, in which 
Fig. 3A shows the procedure performed by a terminal 
on the transmitting side and Fig. 38 the procedure 
performed by a terminal on the receiving side. 

According to the flowchart shown in Fig, 3A, the 
terminal on the transmitting side according to this em- 
bodiment reads moving-picture data into the RAM 13 
via the input portion 11 at step SI 2. Next, the terminal 
on the transmitting side accepts a plurality of effect 
designations, such as a superimposing designation or 
a designation of the transition effect (fade out, wipe, 
etc.), at step 812. Next, the transmitting terminal 
sends effect-designating data (command data), 
which has been accepted at step SI 2, to the commu- 
nication line 101 at step SI 3. This is followed by step 
S14, at which .the transmitting terminal codes the 
moving-picture data by the coding/decoding circuit 
15, and then by step SI 5, at which the coded data is 
sent to the communication line 101 via the interface 
17. It should be noted that an arrangement may be 
adopted in which the moving-picture data is coded in 
advance and the effect-designating data is attached 
thereto before the moving-picture data is sent to the 
communication line 101. 

More specifically, in this embodiment, the effect- 
designating data and the coded, unedited moving- 
picture data are transmitted. 

Next, according to the flowchart shown in Fig. 
38, the terminal on the receiving side receives the ef- 
fect-designating data via the interface 17 at step SI 6 
and stores this data in the RAM 1 3. The receiving ter- 
minal then receives the coded data of the moving- pic- 
ture data via the interface 17 at step SI 7 and stores 
this data in the RAM 13. This is followed by step SI 8, 
at which the receiving terminal decodes the coded 
data, which has been stored in the RAM 13, by the 
coding/decoding circuit 15 and then stores the mov- 
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ing-picture data obtained by decoding in the RAM 13. 
Next, at step S19, the terminal on the receiving side 
edits the moving-picture data, which has been stored 
in the RAM 13, based upon the effect-designating 
data stored in the RAM 13. 5 

The editing operation may be performed as fol- 
lows, by way of example: 

Assume that the rate of the received moving-pic- 
ture data is five frames per second, and that the dis- 
play rate of the display 16 is 20 frames per second. If io 
command data for performing fade-out has been re- 
ceived, fade-out processing is executed after per- 
forming interpolation between frames of the moving 
picture of five frames per second and creating mov- 
ing-picture data of 20 frames per second (Fig. 6A). 15 
The simplest example of interpolation processing 
here is merely to repeat the same picture for four 
frames. Note that moving-picture data according to 
the conventional processing is as shown in Fig. 6B. 

Furthermore, besides fade-out, a transition ef- 20 
feet such as wiping also can be carried out through a 
method similar to that described above. 

Finally, at step S20, the terminal on the receiving 
side displays the moving picture on the display unit 
16. 

Thus, according to this embodiment of the inven- 
tion, after the effect-designating data and moving- 
picture data have been received at the terminal on the 
receiving side, the moving-picture data obtained by 
decoding the coded data is edited based upon the ef- 
fect-designating data. 

When it is attempted to superimpose characters, 
for example, upon a moving picture in the example of 
,4J3^g,g|jig^^ 

which line-drawing data (i.e., a character) has been 35 
superimposed is coded. In a well-known moving-pic- 
ture coding method (e.g., a coding method according 
to the MPEG) the main purpose of which is to com- 
press a natural image, compression cannot be per- 
formed efficiently and, hence, there Is an increase in 40 
the amount of coded data, when it is attempted to im- 
prove picture quality. On the other hand, when it is at- 
tempted to raise the compression rate, there is a 
marked decline in picture quality. For example, it may 
become difficult to read the characters. Furthermore, 45 
when fade-out of a moving picture is performed in a 
case where the moving- picture data is transmitted at 
a low frame rate of ten frames per second in the prior 
art, a change in the image to be faded out also is ex- 
pressed at a low frame rate. As a result, a jerky, crude 50 
effect is alt that is obtained. 

By contrast, in accordance with the present em- 
bodiment, the superimposing of characters is carried 
out immediately before the display of the image in the 
terminal on the receiving side. As a result, there is no 55 
decline in compression efficiency and no deteriora- 
tion in picture quality. Further, in a case where fade- 
out of a moving picture is performed according to this 
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embodiment, it is easy to make the frame rate solely 
of the changing portion of the image to be faded out 
higher than that of other portions by performing the 
above-described interpolation processing in the ter- 
minal on the receiving side. This makes it possible to 
obtain smooth fade-out without any deterioration in 
the transmission efficiency of the image data. 

In the foregoing description and in the drawings, 
an example is described in which coding/decoding of 
a moving picture is carried out by the coding/decod- 
ing circuit 15. However, the embodiment is not limited 
to this arrangement. For example, the coding/decod- 
ing of the moving picture may be realized by software 
processing executed by the CPU 12. Though proc- 
essing speed will decline in such case, the cost of the 
terminal can be reduced. 

Further, in the foregoing description and in the 
drawings, an example is described in which moving- 
picture data entered by the input port 11 or coded data 
received by the interface 17 is stored in the RAM 13 
as is. However, the embodiment is not limited to this 
arrangement. For example, by providing a sufficiently 
fast coding/decoding circuit, coding may be per- 
formed in real time while the moving-picture data is 
being entered or decoding may be performed in real 
time while coded data is being received, with the data 
then being stored in the RAM 13. In this case, the 
processing speed can be raised or, in the former 
case, the memory capacity of the RAM 13 can be re- 
duced. 

Further, in the foregoing description and in the 
drawings, an example is described in which a RAM is 
used for the storing of data. However, the em bod i- 

^^p.^R!t5isiiltQMir^^^^^ 

if high-speed processing is not required, an external 
storage device such a hard disk may be employed. In 
such case, a large quantity of moving-picture data 
can be processed at low cost. 

Thus, in accordance with the embodiment as de- 
scribed above, moving- picture data is edited in a ter- 
minal on the receiving side, whereby a high-quality 
moving-picture transmission can be realized without 
inviting a decline in compression efficiency or a dete- 
rioration in picture quality by coding/decoding. 

It should be noted that the terminal according to 
this embodiment may be a system composed of a 
plurality of devices, e.g., a system of video cameras 
or host computers, or an apparatus comprising a sin- 
gle device, e.g., a host computer in which a moving 
picture is stored. 

Further, the processing according to this em- 
bodiment may be realized by supplying a program, 
which has been stored on a medium, to a system or 
apparatus. 

Thus, in accordance with the first embodiment of 
the present invention, there can be provided a mov- 
ing-picture transmitting method for transmitting infor- 
mation for editing moving-picture data, coding the 
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moving- picture data efficiently and then transmitting 
the coded data. 

Further, in accordance with the first embodiment 
of the present invention, there can be provided a mov- 
ing-picture transmitting method for editing moving- 
picture data, which has been obtained by decoding 
coded data that has been received, based upon the 
received editing information, 

<Second Embodiment> 

According to this embodiment of the invention, 
the moving-picture processing method of the first 
embodiment described above is applied to a techni- 
que in which text is superimposed on a picture in syn- 
chronism with a moving picture, as in the case of cap- 
tions superimposed on a movie or the words to a song 
superimposed on a moving picture in a karaoke sing- 
along system. 

In this embodiment, moving-picture data and text 
data such as captions are held separately. At play- 
back, the text such as captions is developed into a bit- 
map and superimposed on the picture. As a result, it 
is possible to prevent a deterioration in picture quality, 
at the time of image compression/decompression, 
caused by the insertion of text such as captions. At 
the same time, the above-described problems of op- 
erability are solved. 

The second embodiment of the present invention 
will be described with reference to the drawings. 

Fig. 4 is a block diagram showing a moving-pic- 
ture processing system according to this embodi- 
ment. The system includes a memory 111 for storing 
caption text data, a memory 112 for storing moving- 
picture data, a moving-picture compressing circuit 
113, a multiplexer 114, a store-and-forward transmis- 
sion system 115, a demultiplexer 116. a moving-pic- 
ture decompressing circuit 117, a caption superim- 
poser 118, a display unit 119 and signal lines 120, 
121. 

Moving-picture data that has been accumulated 
in the memory 112 is compressed by the moving-pic- 
ture compressing circuit 113, and this data is com- 
bined with text data, which has been accumulated in 
the memory 111, in the multiplexer 114. The com- 
bined data is sent from the multiplexer 114 to the 
store-and-forward transmission system 115. The 
store-and-forward transmission system 115, which 
may be packaged media such as a tape or CD-ROM 
or a communication system such as an ISDN or TV 
broadcast, is used to deliver data to the location of the 
user. The data accepted by the user is separated into 
compressed-image data and text data by the demul- 
tiplexer 116. The image data is sent to the moving-pic- 
ture decompressing circuit 117 through the signal line 
1 20. The text data is sent to the caption superimposer 
118 through the signal line 121 , and the superimpos- 
er 118 develops the text data into a bit-mapped text 



image and superimposes the bit-mapped text image 
on the moving picture decompressed by the moving- 
picture decompressing circuit 117. The thus obtained 
moving picture having captions is displayed by the 
5 display unit 119, It should be noted that the moving 
picture can be obtained as hard copies frame by 
frame using an image forming apparatus (such as a 
laser printer) instead of the display unit. Further, it is 
possible to print out only the captions. 

10 A noteworthy feature of this embodiment is that 

caption superimposer 118 is located on the side of the 
user. In addition, by employing a control panel 122, 
the user can set display/non-display of captions, the 
color of the caption display and its position on the 

15 screen as well as the size of the characters. On the 
basis of the settings made, a control circuit 123 con- 
trols the superimposing operation by means of the 
caption superimposer 118. 

With the conventional transmission method de- 

20 scribed earlier, the caption superimposer is located 
on the transmitting side (the side of the producer of 
the moving picture), and therefore captions are al- 
ways displayed at a fixed position. By contrast, in ac- 
cordance with this embodiment, the user applies 

25 commands to the caption superimposer so that it is 
possible to freely set display/non-display of captions, 
the color of the caption display and its position on the 
screen as well as the size of the characters. This rep- 
resents a great improvement in operability. Further- 

30 more, in accordance with this embodiment, it is pos- 
sible to provide the excellent picture quality and com- 
pression efficiency described above. 

Further, an MPEG encoder, which is a moving- 
picture compression system in accordance with inter- 

35 national standards, can be used as the moving-pic- 
ture compressing circuit 113 shown in Fig. 4. Since a 
"user data areaj* is defined in the MPEG bit stream 
and any data can be written in for each picture in con- 
formity with the MPEG, the multiplexer 1 14 performs 

40 an operation through which the text data is written in 
the "user data area". The demultiplexer 116 reads out 
text data from the "user data area" of the MPEG bit 
stream and sends it to the caption superimposer 118. 
The moving- picture decompressing circuit 117 uses 

45 an MPEG decoder. 

In accordance with this embodiment, an advan- 
tage is that it is possible to generate coded data that 
is in conformity with the MPEG. 

In a case where data is reproduced using an or- 

50 dinary MPEG decoder instead of the method of repro- 
duction described In this embodiment, the user data 
area is ignored so that captions simply are no longer 
capable of being displayed. The moving picture, how- 
ever, can be still reproduced correctly. Thus, compa- 
ss tibility is maintained. 

In accordance with the second embodiment of 
the invention as described above, it is possible to 
avoid a deterioration in picture quality caused by text 
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being superimposed on the picture. In addition, the 
user is capable of freely selecting the method of dis- 
playing captions. 

Fig. 5 is a block diagram showing a moving-pic- 
ture processing system according to a third embodi- 5 
ment. The system includes memories 131, 132, 133 
for storing caption text data, a memory 1 34 for storing 
a moving picture, a moving-picture compressing cir- 
cuit 1 35. a multiplexer 136, a store-and-forward trans- 
mission system 137, a demultiplexer 138, a moving- io 
picture decompressing circuit 139, a caption superim- 
poser 1 40, a display unit 1 41 and a changeover switch 
142. 

The memories 131, 132, 133 contain different 
types of text data. For example, assume that the 15 
memories 131, 132, 133 contain captions in Japa- 
nese, English and Chinese, respectively. These three 
types of text data are sent to the multiplexer 136, 
which superimposes the data on image data obtained 
from the memory 134 and then compressed via the 20 
moving -picture compressing circuit 1 35. The resulting 
data is sent to the store-and-forward transmission 
system 137. 

On the receiving side, the data is passed through 
the demultiplexer 138 to separate it into the com- 25 
pressed image data and the three types of text data. 
The compressed image data is decompressed by the 
moving-picture decompressing circuit 139 and the 
decompressed data is then sent to the caption super- 
imposer 140. Any one of the three types of text data 30 
is selected by the switch 142. According to this em- 
bodiment, the user is capable of selecting a caption 
in the desired language from among the captions in 
^t.bigp eil g| qigq gs^t^ ^ - ^ " 

into a bitmap by the superimposer 140 and superim- 35 
posed upon the moving picture, with the result being 
displayed on the display unit 141. 

By adopting the arrangement described above, 
the user is capable of selecting and displaying any 
caption of a plurality of types. 40 

Though the types of caption text are made avail- 
able in three systems, it goes without saying that this 
does not constitute a limitation. Caption text of types 
In any number of systems can be entered to the extent 
allowed by the capacity of the store-and-forward 45 
transmission system and the transfer capability. In 
addition, it is not necessary to prepare text In a num- 
ber of languages, and it goes without saying that text 
having different content in the same language can be 
entered as well. 50 

Instead of developing text into a bitmap and su- 
perimposing It on an image, an arrangement may be 
adopted in which text is read by a voice synthesizer 
and either mixed with audio that accompanies amov- 
ing picture or substituted for the audio before play- 55 
bacl< is performed. 

As many apparently widely different embodi- 
ments of the present invention can be made without 



departing from the spirit and scope thereof, it is to be 
understood that the invention is not limited to the spe- 
cific embodiments thereof except as defined in the 
appended claims. 



Claims 

1. Amoving picture processing apparatus compris- 
ing: 

receiving means for receiving coded digital 
moving-picture data and command data for com- 
manding effect processing to be applied to the 
digital moving-picture data; 

decoding means for decoding the coded 
digital moving-picture data; 

processing means for subjecting the digi- 
tal moving-picture data decoded by said decod- 
ing means to effect processing; and 

image forming means for forming an im- 
age in dependence upon the digital moving-pic- 
ture data processed by said processing means; 

said digital moving-picture data and said 
command data being transmitted by a common 
transmission line. 

2. The apparatus according to claim 1, wherein the 
frame rate of the moving picture formed by said 
image forming means is greater than the frame 
rate of the moving picture received by said receiv- 
ing means. 



3. The apparatus according to claim 2, wherein said 
pro^'^ingMm'e'ans%xeci3te''s'^Tramfe^intiBr|^ 
processing. 

4. The apparatus according to claim 3, wherein said 
processing means subjects a moving picture that 
has undergone frame Interpolation processing to 
a transition effect. 

5. The apparatus according to claim 1 , wherein said 
processing means executes superimposing proc- 
essing. 

6. The apparatus according to claim 5, wherein said 
command data includes text data of a plurality of 
types. 

7. The apparatus according to claim 6, further com- 
prising selecting means for selecting, from among 
the text data of the plurality of types, text data 
that is to be superimposed. 

8. A moving picture processing method comprising 
the steps of: 

receiving coded digital moving-picture 
data and command data for commanding effect 



6 



11 EP 0 596 732 A1 12 

processing to be applied to the digital moving-pic- 
ture data; 

decoding the coded digital moving-picture 

data; 

subjecting the decoded digital moving-pic- 5 
ture data to effect processing; 

forming an image in dependence upon the 
processed digital moving-picture data; and 

transmitting said digital moving-picture 
data and said command data by a common trans- io 
mission line. 

9. An apparatus for receiving digitally coded video 
signals, wherein scene transition effects and/or 
captioning effects are performed upon received is 
image data in accordance with command data re- 
ceived with said image data. 

10. An image signal or recording, wherein digitally 
coded motion image data is accompanied by 20 
command data defining image transition effects 

to be used in the reproduction of said motiion im- 
age data. 
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